Integrating Association Rule Mining Algorithms with Relational Database Systems

Mining for association rules is one of the fundamental data mining methods. In this paper we describe how to efficiently integrate association rule mining algorithms with relational database systems. From our point of view direct access of the algorithms to the database system is a basic requirement when transferring data mining technology into daily operation. This is especially true in the context of large data warehouses, where exporting the mining data and preparing it outside the database system becomes annoying or even infeasible. The development of our own approach is mainly motivated by shortcomings of current solutions. We investigate the most challenging problems by contrasting the prototypical but somewhat academic association mining scenario from basket analysis with a real-world application. We thoroughly compile the requirements arising from mining an operative data warehouse at DaimlerChrysler. We generalize the requirements and address them by developing our own approach. We explain its basic design and give the details behind our implementation. Based on the warehouse, we evaluate our own approach together with commercial mining solutions. It turns out that regarding runtime and scalability we clearly outperform the commercial tools accessible to us. More important, our new approach supports mining tasks that are not directly addressable by commercial mining solutions.